Value in Health
○ Elsevier BV
Preprints posted in the last 7 days, ranked by how well they match Value in Health's content profile, based on 11 papers previously published here. The average preprint has a 0.01% match score for this journal, so anything above that is already an above-average fit.
Gomez-Guillen, D.; Fernandez-Lopez, D.; Cerquides, J.; Arcos, J. L.; Diaz, M.
Show abstract
Background: Cost-effectiveness analyses (CEA) built on simulation models are usually delivered through project-specific scripts. Each new model, and often each new question, rebuilds the same scaffolding: parameter management, calibration, base-case analysis, sensitivity analysis and export. This is costly, and is where traceability between an estimate and the model version producing it is lost. We present THALASSA, a web application providing this infrastructure once, for any model satisfying a small programming contract. Methods: A simulation model is connected by one file implementing a fixed contract of R functions that determine its behaviour. THALASSA exposes the complete CEA workflow in a browser: a generated overview of the connected model, parameter editing, calibration, base-case comparison with the efficiency curve, and deterministic and probabilistic sensitivity analysis at a willingness-to-pay threshold, each with timestamped file export carrying the parameter values it was run with. Every session is bound to a repository version, so results carry their generating model version. Results: We demonstrate THALASSA on two public models of contrasting difficulty: a simple tutorial model and a generic cancer screening model with two competing natural-history pathways and a 21-dimensional calibration deliberately built to be more complex. Both ran on the same unmodified setup, implementing the same contract, with no code changes. Conclusions: Separating analysis infrastructure from model logic makes the CEA workflow reusable across projects and reproducible by construction. A modelling team can move from a versioned repository to audit-ready, exportable CEA outputs without writing analysis code. The integration effort does not grow with model size and one installation serves unrelated projects.
de Araujo, F. A. S.; Ikawa, J. N. R.; Lima, A. d. S.; Bernz, L. M.; CORREIA, L. R.
Show abstract
Background: Lean Healthcare has been adopted as an approach to reducing waste, enhancing process efficiency, and optimizing clinical and administrative workflows. However, empirical evaluations aimed at establishing a causal relationship between Lean Healthcare interventions and outcomes remain limited. This evidence gap is particularly pronounced in the context of large-scale implementations in low- and middle-income countries, where more efficient allocation could yield substantial improvements in population health. Methods: We conducted an impact evaluation study employing a staggered difference-in-differences approach to estimate the effects of the Lean Healthcare in Emergency Departments (ED) initiative in the Brazilian context. The intervention was implemented between January 2017 and December 2025 across 309 hospitals. We analyzed hospital-level indicators of the number of hospitalizations and the average length of stay (in days) among patients admitted through urgent or emergency pathways. Results: Our estimates suggest that the intervention was associated with an average increase of 38.6 admissions per hospital per month (p = 0.033), corresponding to approximately a 5.9% increase relative to the monthly admission volume. We also observed a reduction of 0.184 days in the average length of stay (p = 0.058), equivalent to an approximately 3.0% decrease in hospitalization time. These effects exhibited distinct temporal dynamics: the effect on the number of hospitalizations emerged gradually and increased with the duration of the intervention, whereas the reduction in length of stay became evident earlier. Conclusion: This study provides evidence of a causal effect of Lean Healthcare implementation in emergency departments on operational performance. The simultaneous increase in admissions and reduction in length of stay suggests improved bed productivity and effective capacity without expanding physical infrastructure. These findings support the use of flow-management methodologies as an effective strategy for mitigating emergency department overcrowding while increasing service capacity.
Karalis, V.
Show abstract
The aim of this study was to develop a framework for assessing the incremental value of decisions on drug amount, timing, input structure, cessation and restart. The LOGOS (Longitudinal Optimization of Global Outcomes and Strategies) approach combines matched nested regimen classes, mechanism-specific benchmarks and evidence requirements in a single longitudinal comparison. Thus, the contribution of additional regimen freedom is assessed separately from improvements due to different resources, constraints or evaluation horizons. Nested feasible classes have nondecreasing objective suprema, whereas a strict gain requires a feasible witness above a valid bound for the smaller class. The approach is illustrated through analytical examples of periodic smoothing, matched desensitization, relaxed exposure allocation, finite-course loading and safety after cessation. In a normalized finite Emax model, integrated benefit increased from 0.629606 for three equal boluses to 0.672945 after optimization of amounts, 0.677043 for an amount-and-time candidate and 0.679553 for a globally certified loading-maintenance-coast solution. The adjacent observed gains were 6.88%, 0.61% and 0.37%, respectively, and all schedules satisfied the full-tail injury bound. In addition, a periodic factorial example separated response curvature from a joint recovery and repair process, including a null case and results dependent on feasibility constraints. The LOGOS framework allows assessment of whether additional regimen flexibility improves benefit, provides no added value or restores feasibility. These dimensionless examples support conditional mathematical and model-based conclusions; clinical superiority and safe restart in patients have not been established.
Srivastava, S.; Ziegler, S.; Parmar, D.; Strupat, C.; De Allegri, M.
Show abstract
Objectives: Women's empowerment is a key determinant of health and well-being and may shape how women access and benefit from publicly-funded health insurance (PFHI). We aimed to examine whether different levels of women's empowerment were associated with 1) verification in India's PFHI Pradhan Mantri Jan Arogya Yojana (PM-JAY), 2) outpatient care utilization, and 3) resultant out-of-pocket expenditure. Design: We conducted a cross-sectional observational study. We analyzed data from a household survey, created a composite empowerment index using structural equation models, and used multivariate regression models to test associations between women's empowerment and the outcome variables. Setting: The study was conducted between December 2019 and February 2020 among PM-JAY eligible households in six Indian states (Bihar, Chhattisgarh, Gujarat, Meghalaya, Tamil Nadu, and Uttar Pradesh). Participants: 14,378 women across the six states. Results: Three main findings emerged: First, high levels of women's empowerment were associated with higher odds of completing PM-JAY verification (AOR: 1.22, 95% CI: 1.09-1.36). Second, women's empowerment was not associated with outpatient service utilization. And third, either medium levels of empowerment (coef = -0.49, p = 0.020) or PM-JAY verification (coef = -0.37, p = 0.034) was associated with lower OOPE for outpatient services. Women who had both, PM-JAY verification and medium levels of empowerment, experienced higher OOPE (coef = 0.45, p = 0.058). Conclusions: Women's empowerment is an important factor related with PM-JAY verification, but not with outpatient service use. OOPE varied depending on whether women were both empowered and PM-JAY verified or not. Our results suggest that institutional and structural barriers may play a greater role in healthcare access than individual factors pertaining to empowerment alone. Achieving equitable access therefore requires gender-responsive systems that reduce administrative barriers and support women in navigating the healthcare pathway. Our findings highlight the importance of integrating outpatient care into PM-JAY to increase financial protection, streamlining verification processes to improve women's engagement with PM-JAY and the broader health system, and advancing health financing reforms that better support women.
Smith, J.; Smith, B.; Pittman, T.
Show abstract
This paper examines the impact of Cancer Care Ontario's regionalization of lung cancer surgery on blood transfusion use for thoracic surgery at The Ottawa Hospital. Using aggregate data scraped from the paper ''Changing trends in blood transfusion: an analysis of 244,013 hospitalizations'' by Shehata et al. (2014) we estimated the effect regionalization on the log mean number of blood transfusions among thoracic surgeons (who treat lung cancer), utilizing other surgical specialties as controls. We found that the average number of blood transfusions increased significantly among thoracic surgeons following the opening of a new cancer care centre at the hospital in December 2009, while control specialties showed no parallel change. Contextualizing these results in a modified ''population level'' treatment threshold model, it is argued that the main driver of this increase in blood transfusions among thoracic surgeons was due to the influx of less skilled thoracic surgeons to The Ottawa Hospital, and the closing of thoracic surgery units at smaller hospitals which was part of the broader regionalization process.
Arojo, O.; Jones, A. M.
Show abstract
Background: Epidermal Growth Factor Receptor (EGFR) tyrosine kinase inhibitors (TKIs) are widely used for EGFR-mutated non-small cell lung cancer (NSCLC) but have distinct toxicity profiles across generations. Aims: To investigate relationships between EGFR-TKI physicochemical, pharmacokinetic and pharmacological properties and drug-specific adverse drug reactions (ADRs) using national and global pharmacovigilance data. Methods: ADRs for erlotinib, afatinib and osimertinib were extracted from the UK MHRA Yellow Card scheme and WHO VigiAccess. Disproportionality analysis used reporting odds ratios (RORs). Signals were defined as ROR >2, lower 95% confidence interval >1 and >3 cases. Drug properties were curated from ChEMBL, emc, SmPC and EMA sources. Results: VigiAccess contained 88,662 individual case safety reports (163,619 ADRs) and Yellow Card contained 752 reports (1,814 ADRs). Four System Organ Class (SOC) signals were identified in VigiAccess and three in Yellow Card, with five additional Yellow Card Preferred Term signals. Eye disorders showed the strongest consistent association with erlotinib (VigiAccess ROR 4.33; Yellow Card ROR 4.18). Osimertinib was associated with pneumonitis (ROR 7.67) and pulmonary embolism (ROR 3.94). Afatinib showed psychiatric disorder signals in both databases (ROR 3.2-3.3) and diarrhoea (ROR 2.79). Neoplasm-related signals likely reflected confounding by indication. Conclusions: EGFR-TKIs demonstrate generation-specific pharmacovigilance profiles. Integration with drug properties supports mechanistic plausibility for several signals, while the atypical afatinib psychiatric association warrants further investigation. Findings are hypothesis-generating and require confirmation in controlled observational or mechanistic studies.
Chen, Y.-w.; Bhunyanaphakul, V.; Kosago, P.; Finkelman, M.; Lee, D. J.
Show abstract
Purpose: To compare the performance of three large language models (LLMs), ChatGPT-o1, Grok 3, and Gemini 2.0 Pro, on the American College of Prosthodontists' National Prosthodontic Resident Examination (NPRE). Materials and Methods: Official NPRE examinations from years 2015-2024 were retrieved from the American College of Prosthodontists website and used to prompt each LLM. Paid access tiers and advanced reasoning were enabled. Responses from LLMs were recorded and compared against the official answer keys for accuracy, variability across years, and consistency of responses. Duplicate questions were retained for the primary analysis and removed in a sensitivity analysis. Accuracy (percent correct) was compared using generalized estimating equations binary logistic regression (primary); Cochran's Q with Bonferroni-adjusted McNemar tests (sensitivity). Year-to-year variability (SD of annual accuracy) was compared using weighted Levene tests. Consistency across duplicate items was quantified as the percentage of perfect matches and compared using Cochran's Q with Bonferroni-adjusted McNemar tests. Results: Across 1,487 questions, overall accuracy differed among models (global P < 0.001). ChatGPT-o1 achieved 78.4% correct, Grok 3 76.4%, and Gemini 2.0 Pro 73.2%; ChatGPT-o1 and Grok 3's accuracy significantly exceeded Gemini 2.0 Pro (both P [≤] 0.016), with no significant difference between ChatGPT-o1 and Grok 3 (P = 0.099). After removing duplicates (n = 1,200), results were similar: ChatGPT-o1, Grok 3, and Gemini 2.0 Pro were 77.1%, 76.6%, and 71.5% correct, respectively; ChatGPT-o1 and Grok 3's accuracy was significantly higher than Gemini 2.0 Pro (both P < 0.001). Variability across years did not differ significantly (P [≥] 0.097). Grok 3 showed significantly higher perfect-match consistency (79.2%) than ChatGPT-o1 (69.4%) and Gemini 2.0 Pro (64.5%) (P [≤] 0.010). Conclusions: Contemporary LLMs showed 70-80% accuracy on postgraduate prosthodontic multiple-choice content. While promising for educational support, residual error and inconsistency warrant human oversight and transparent methods reporting before curricular adoption.
Bergman, H. I.; Liu, V.; Ali, S.
Show abstract
Background: Ambient artificial-intelligence (AI) scribes are being deployed beyond medicine into allied-health practice, but comparative evidence mainly concerns physician documentation. Objective: To compare quality and transcript-grounded clinical-risk error burden of AI-generated and allied health professional notes from identical simulated consultations. Methods: Paired simulation across dietetics, physiotherapy, occupational therapy and speech and language therapy (five consultations each; 20 pairs; 40 notes). Heidi generated an unedited first-draft note and a professional documented each encounter. Professionals rated PDQI-9 quality. Professional and automated review identified unsupported statements and omissions; two language models graded clinical risk. Co-primary outcomes were PDQI-9 score and Critical+High error density. Results: Mean PDQI-9 scores were 38.45 and 35.05 for AI and professional notes (paired difference +3.40, 95% CI +0.01 to +6.79; Wilcoxon p=0.042; dz=0.47). The Bayesian estimate was +3.47 (94% equal-tailed credible interval +0.66 to +6.30; P[AI>professional]=0.989). Automated-reviewer all-error density was 3.35 vs 8.55 per note (professional-minus-AI difference 5.20, 95% CI 3.85 to 6.55); omission density was 1.45 vs 5.20 (difference 3.75, 2.80 to 4.65). Critical+High density was 0.35 vs 0.75 (difference 0.40, -0.10 to 1.00). The pooled quality result was driven principally by dietetics. Conclusions: AI-generated notes achieved modestly higher pooled quality and fewer transcript-grounded errors, chiefly omissions; serious-error evidence was inconclusive. Small discipline samples and incomplete ground truth preclude safety or discipline-wide superiority claims.
Tudball, M. J.; Cirulli, E. T.; Holmes, M. V.; Davey Smith, G.; Richards, J. B.
Show abstract
Recent reanalyses of major lipid-lowering trials have reported substantially greater treatment benefit in individuals with higher polygenic risk scores (PRSs) for coronary artery disease (CAD). We tested whether trial eligibility criteria could generate this pattern through collider bias. In simulations where the true PRS-by-treatment interaction was set to zero, we reproduced apparent PRS-dependent treatment effects similar to those observed in reanalyses of three trials. This included FOURIER-like hazard ratios for lipid-lowering treatment versus placebo of 0.77 in the high CAD PRS stratum and 0.95 in the low CAD PRS stratum. In the Helix Research Network (n = 175,106) and UK Biobank (n = 92,859) cohorts, where we did not apply the trial eligibility criteria, the CAD PRS showed little evidence of modifying the effect of statin use on acute coronary events. We then used LDL cholesterol response as a better-powered test of collider bias: we found no evidence that LDL PRS modified statin response in population cohorts, yet stratifying on baseline LDL cholesterol induced apparent PRS-dependent response matching the simulations. Importantly, treatment randomization does not protect against eligibility-induced collider bias. PRS-dependent response may therefore reflect bias from eligibility rather than biology. These results should inform the planning and execution of clinical trials for precision medicine.
Krutsch, K.; Appiah, D.
Show abstract
Purpose Prescribing to lactating women is constrained by limited evidence. Published compatibility classifications provide clinical guidance and a discrete research comparator. How often prescribing diverges from that guidance, and what characteristics are associated with divergence, is unknown at national scale. Methods Retrospective analysis of prescriptions filled during lactation, identified in the 2012-2014 MarketScan Commercial Claims Database. Prescriptions were classified as concordant (Hale's lactation risk category L1-L3) or discordant (L4-L5). Multivariable logistic regression identified factors associated with concordance. Results Of 96,484 new prescriptions, 95,710 (99.2%) were concordant and 774 (0.8%) were discordant. Discordant prescriptions were filled later after delivery (median 59 vs 33 days) and had a day supply three times longer (30 vs 10 days). In multivariable analysis (N=28,732), concordant prescriptions were associated with maternal age under 35 (adjusted odds ratio (aOR)=1.34, 95% confidence interval (CI): 1.02-1.75) and chronic-condition medications (aOR=10.94, 95% CI: 6.47-18.50), and were less likely within 14 days of delivery (aOR=0.53, 95% CI: 0.38-0.73) and with each additional day of supply (aOR=0.96, 95% CI: 0.96-0.96). Compared with family practice, concordance did not differ for obstetric, internal medicine, or emergency medicine prescriptions, but prescriptions from other provider types had lower odds of concordance (aOR=0.57, 95% CI: 0.41-0.79). Conclusions Prescribing during lactation was overwhelmingly concordant with published compatibility guidance; the rare divergence was patterned by prescription and prescriber characteristics. Because the categories are an expert classification rather than consensus guidance, divergence should not be read as inappropriate prescribing.
Tuffaha, M. Z.; Dick, D. W.; Li, J.; Heffernan, J. M.
Show abstract
We develop an age- and immunity-structured COVID-19 model with setting-specific contacts, calibrated to Ontario incidence and hospital admissions before Omicron. We simulate counterfactual vaccination strategies and contact restrictions in schools, workplaces, and other community settings, evaluating projected severe-case demand against an illustrative healthcare-capacity benchmark. Vaccination combined with workplace and community contact restrictions keeps simulated demand within capacity. In contrast, relaxing restrictions in either setting produces large surges and repeated capacity threats, even when school restrictions remain. School-only changes produce smaller effects on severe disease. These findings highlight the value of vaccination and setting-specific contact restrictions that prioritize workplace and community settings for pathogens with adult-concentrated severe outcomes.
Yin, D. E.; Zivich, P. N.; Giganti, M. J.; Wu, D.; Edwards, J. K.; Cole, S. R.
Show abstract
The efficacy of current antiretroviral interventions for prevention of vertical HIV transmission is both a triumph and a challenge for future efficacy studies of novel agents, including long-acting antiretroviral agents. Given the effectiveness of current antiretroviral interventions, standard randomized controlled trials to assess novel interventions targeted towards further reducing vertical HIV transmission may be infeasible due to required sample sizes. To address this study design challenge, we propose a novel augmented clinical trial design, where a randomized trial with an active comparator is augmented with a concurrent observational cohort to supplement the standard-of-care arm. The proposed design and analysis can overcome sampling biases, and an associated diagnostic can evaluate potential biases from unmeasured covariates. The proposed trial design is illustrated with an example and studied further using a simulation study. In the illustrative example, an augmented trial with a 3:1 allocation ratio in the trial component yielded an unbiased treatment effect estimate, as compared to the trial component alone, and the standard error was reduced substantially. Although the total sample size of the augmented trial was larger than a similarly powered 1:1 standard trial, the sample size of the augmented trial's trial component was half the size of the standard trial and within the range of historically similar trials, thereby providing both feasibility and potential cost advantages. Although limitations with this design remain, many of these limitations can be mitigated by careful study planning and coordination. Augmented trials provide a potentially applicable solution to studying the efficacy of long-acting antiretrovirals for infant postnatal prophylaxis against HIV.
Lefebvre, A.; Benusiglio, P.; Nuel, G.
Show abstract
Background: Current Lynch syndrome (LS) prediction models do not account for detailed family structure and tumour mismatch repair (MMR) status. We developed LynchRisk, a Bayesian model that estimates individualized LS probability from pedigree, cancer history and tumour MMR test results. Methods: LynchRisk combines Mendelian inheritance, age- and sex-specific cancer hazards, affected and unaffected relatives, and detailed MMR status. Parameters were derived from the literature. Results: In two simulated cases of probands with LS-associated cancers, family structure and MMR status markedly changed LS probability estimates, with implications for germline testing decisions. LynchRisk also identified situations in which a negative germline result might be considered sufficient to rule out LS or, conversely, prompt further investigations such as somatic MMR or long-read sequencing. Discussion: LynchRisk gives individualized LS probability by combining clinical, family, and tumour data at different stages of the diagnostic process. Further work will focus on recalibrating cancer hazard functions, validating model performance through simulations, and developing an R package for use by interested research groups.
Rothermich, K.; Clark-Stallkamp, R.; Baker-Iyore, R. P.; Cuatico, C.; Lee, J. G. L.; Bobb, S. C.
Show abstract
Objectives: Health professions students often receive limited training on how to adjust their communication for patients whose first language is not English (LX), despite the risks for diagnosis, treatment, and patient trust. Based on Communication Accommodation Theory, we developed a brief module to improve students' awareness and knowledge of strategic communication adjustments for LX patients. Methods: We tested feasibility in a single-group, pre-post pilot study guided by the Kirkpatrick framework with undergraduate (n=22) and graduate (n=41) speech-language pathology and psychology students in Massachusetts and North Carolina. The module included instructional content, video and audio examples, interactive activities, and a podcast. Learning was assessed with knowledge checks, and self-reported facilitative and resistant accommodation with the Communication Accommodation Behavior Scale (CABS). After training, students rated their confidence and likelihood of using accommodation strategies. Results: At both sites, facilitative accommodation increased and resistant accommodation decreased significantly, with a larger decrease at the undergraduate site. Knowledge-check accuracy was high at both time points, suggesting a ceiling effect. Students reported moderate- to-high confidence in adapting their speech and high likelihood of using a learned strategy in future clinical encounters. Conclusions: A brief module can be feasibly delivered across institutions and is associated with short-term improvements in self-reported communication accommodation. Practice implications: We recommend future work evaluate whether these changes translate into clinical behavior long-term and refine knowledge-check items to better capture learning gains.
Habib, M. H.; Mayeda, E. R.; Barry, L.; Arah, O. A.; Nau, C. L.; Zhou, H.; Kezios, K. L.; Nianogo, R. A.
Show abstract
Financial well-being (FWB) encompasses objective and subjective financial status and may influence health. We developed and validated a parsimonious FWB index using data from 23,006 Health and Retirement Study (HRS) participants aged 50 years and older from 2006 to 2022. The FWB-5 Index was derived using correlated two-factor confirmatory factor analysis accounting for household clustering. Indicators included household wealth, household income-to-poverty ratio, monthly financial strain, satisfaction with one's financial situation, and perceived financial mastery. We assessed convergent and discriminant validity using the Consumer Financial Protection Bureau Financial Well-Being (CFPB-FWB) Scale and Satisfaction with Life Scale, respectively. Associations with pre-baseline socioeconomic factors and subsequent depressive symptoms and health conditions were examined to assess construct and predictive validity. Model fit indices supported the two-factor structure (robust comparative fit index = 0.99; robust root mean square error of approximation = 0.05). Correlations were stronger with the CFPB-FWB Scale (Spearman P = 0.80) than with life satisfaction (Spearman P = 0.43). Associations with socioeconomic factors and subsequent health outcomes were in the hypothesized directions. These findings support the FWB-5 Index as a parsimonious measure for examining financial well-being and its associations with health across HRS waves, including those predating the CFPB-FWB Scale.
Chang, H.-H.; Chang, C.-K.; Tang, T.-Y.; Jhan, J.-H.; Dong, S.-W.; Juan, Y.-S.; Geng, J.-H.; Ou, H.-C.; Lee, H.-Y.; Li, C.-Y.; Yeh, H.-C.; Leong, M.-K.; Liu, C.-H.; Lee, Y.-C.; Huang, S.-P.
Show abstract
Background and Objective: Adding belzutifan to adjuvant pembrolizumab improved disease-free survival (DFS) in LITESPARK-022, but its durability beyond the 1-year treatment is unknown. Methods: Following a registered plan, we reconstructed individual patient data from published Kaplan-Meier curves of ten phase 3 adjuvant renal cell carcinoma trials. Within each trial, we estimated restricted mean survival time differences ({Delta}RMST), hazard ratios (HRs) before and after planned treatment ended, and DFS differences. Key Findings and Limitations: In LITESPARK-022 (central review), the 24-month {Delta}RMST was 1.34 months (95% confidence interval 0.73-1.95), and the belzutifan arm remained ahead after month 12 ({Delta}RMST increment 0.74 months/yr, 0.32-1.19). The HR was 0.55 (0.43-0.71) before month 12 and 1.16 (0.87-1.54) afterwards (post/pre ratio 2.10); investigator assessment showed similar attenuation. Similar ratios occurred in KEYNOTE-564 at its first interim analysis (2.02), whose benefit persisted, and in SORCE and PROTECT (2.09, 2.59), whose benefit faded. From months 12 to 30 (post hoc), the DFS difference changed by -3.7 percentage points (-8.1 to 0.8) with central review and 0.6 (-4.0 to 5.0) with investigator assessment. Data were reconstructed, and follow-up was short. Conclusions and Clinical Implications: Relative hazard reductions with belzutifan were greater before month 12; durability of the absolute DFS advantage is uncertain and assessment-dependent, so single-HR projections should be avoided.
Nahid, K.; Anam, M.; Saon, M. T. H.; Any, O. H.; Hassan, K. M. R.; Anwar, M. M.
Show abstract
Large language models (LLMs) are increasingly consulted for health advice, but they can produce fluent statements that are false or unsupported, known as hallucinations, and this risk is poorly characterised in low-resource languages such as Bangla. We measured hallucinations of locally deployable open-weight LLMs in Bangla diabetes question answering and tested whether retrieval-augmented generation (RAG) grounded in Bangladeshs National Guideline on Diabetes Mellitus reduces them. Forty patient questions spanning six clinical sub-domains were each answered twice, under zero-shot prompting, by Qwen3.5-9B, TigerLLM-9B, Qwen3.6-35B-A3B and Gemma-4-26B-A4B, and by BanglaDiaRAG, which adds hybrid dense (BGE-M3) and sparse (BM25) retrieval from a physician-reviewed Bangla guideline knowledge base to Qwen3.5-9B. Two physicians rated the 400 de-identified responses by consensus for medical accuracy, completeness, safety, readability and reliability on a 0-1 scale, and responses with accuracy below 0.70 were classified as hallucinated. Hallucination rates were 2.5% for Gemma-4-26B-A4B, 3.8% for TigerLLM-9B, 22.5% for Qwen3.6-35B-A3B and 77.5% for Qwen3.5-9B. Guideline grounding lowered the rate of Qwen3.5-9B to 11.3% (difference -66.3 percentage points; 95% CI -76.3 to -55.0) and raised every criterion by 0.24 to 0.38 points (Holm-adjusted p < 0.001). BanglaDiaRAG outscored Qwen3.6-35B-A3B on all five criteria and matched or exceeded TigerLLM-9B, but it remained below Gemma-4-26B-A4B in safety, readability and reliability, and its residual errors were specific, confident and potentially harmful. Guideline grounding can turn a weak general-purpose model into a much safer Bangla diabetes assistant, but the choice of base model mattered as much as retrieval and no configuration was free of hallucination, so clinical oversight remains essential before patient-facing use.
Wang, X.; Macieira, T. G. R.; Hong, Y.-R.; Revere, L.; Xie, Z.
Show abstract
Objective To examine the association between patient-reported shared decision-making (SDM) and documented advance care planning (ACP) among adults with Alzheimer disease and related dementias (ADRD), and to assess whether this association differed between adults with and without ADRD. Design Retrospective observational study using linked survey and electronic health record (EHR) data. Setting and Participants Participants aged [≥]50 years in the All of Us Research Program who completed a patient-reported SDM measure and had linked EHR records. The primary analytic cohort included 3,635 adults with ADRD; 123,681 adults without ADRD comprised a secondary comparison cohort. Methods SDM was measured by how often clinicians asked participants about their opinions or preferences regarding medical care or treatment and was categorized as high (always/most of the time) or low (some of the time/ None of the time). Documented ACP was identified within 3 years before or after the SDM assessment. Modified Poisson regression with robust variance estimated prevalence ratios (PRs) and adjusted PRs (aPRs). Results Among adults with ADRD, 60.6% reported high SDM. Documented ACP prevalence was 5.1% among participants with high SDM and 3.6% among those with low SDM (P=0.027). High SDM was associated with greater ACP prevalence in the unadjusted model (PR, 1.44; 95% CI, 1.04-1.99); the association remained positive but was attenuated after full adjustment (aPR, 1.28; 95% CI, 0.93-1.77). Among adults without ADRD, high SDM was associated with documented ACP (aPR, 1.17; 95% CI, 1.03-1.33). Conclusions and Implications Among adults with ADRD, high SDM was associated with greater ACP prevalence in unadjusted analyses, although the fully adjusted association was not statistically significant. In contrast, higher SDM remained significantly associated with ACP among those without ADRD. The low prevalence of ACP highlights an important opportunity to expand discussions of current and future care preferences as a routine component of patient-centered care.
Graham, N. S.; Auger, S. D.; Attaallah, B.; Batla, A.; Scott, G.
Show abstract
Large language models (LLMs) are increasingly evaluated for clinical tasks, but safe use requires reliable uncertainty estimates as well as high accuracy. We tested eight general-purpose LLMs on 100 validated neurology board-style multiple-choice questions and compared three confidence signals: self-reported confidence, critique-implied confidence from an independent model instance, and consistency-based confidence from repeated sampling. Accuracy varied from 45% (Llama 3.1 8B) to 92% (Gemini 3 Flash/Pro), but confidence was generally overestimated, particularly in lower-accuracy models. Consistency-based confidence provided the best discrimination between correct and incorrect answers (pooled AUROC 0.74), outperforming self-report, although calibration differences between methods were not significant. Calibration should therefore be reported alongside accuracy when clinical LLMs are evaluated.
Jospe, M. R.; Bohlen, L. C.; Crawshaw, J.; Somerville, J.; LaRegina, E.; Willis, H. J.; Gustafson, S. K.; Schembre, S. M.
Show abstract
Aims: To understand how users believe continuous glucose monitoring (CGM) changes their behaviour, which app features they credit, and whether endorsing more mechanisms of action (MoAs) is linked to reported diet and physical activity change. Methods: We surveyed 112 U.S. adults with type 1 or type 2 diabetes who had used a Dexcom or Abbott Libre CGM within 12 months. Participants rated 26 MoA statements mapped to the Capability, Opportunity, Motivation-Behaviour (COM-B) model, identified contributing app features, and reported CGM-attributed diet and physical activity change. Logistic regression examined MoA count and COM-B domains as predictors of behaviour change. Results: Participants (mean age 54.5 years; 60.7% female) reported dietary (82.1%) and physical activity (46.4%) change and endorsed a median of 18 MoAs, most often relating to knowledge, skills and behavioural consequences. Real-time glucose and graphs were attributed to many MoAs. Each additional MoA was associated with higher odds of dietary (OR 1.27, 95% CI 1.15-1.44) and physical activity change (OR 1.27, 95% CI 1.16-1.43). Motivation made the strongest distinct contribution among COM-B domains. Conclusions: Users perceive CGM as engaging many behavioural mechanisms; endorsing more was linked to behaviour change. Motivation may be a key target for CGM-based behavioural support.